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ABSTRACT 

According  to  the  1972Marine  Mammal  Protection  Act, 
the  National  Marine  Fisheries  Service  has  the 
responsibility  to  monitor  and  protect  marine  mammals 
including  the  great  whales.  To  effectively  implement  this 
act  NMFS  must  routinely  observe  whales  to"  measure  the 
size  and  health  of  their  population.  The  Act  further 
stipulates  that  observations  of  whales  be  done  from  a 
distance  far  enough  away  to  ensure  that  the  whales  not 
be  disturbed.  Thus  a  need  exists  for  an  accurate  method 
to  sense  the  whales  from  an  airplane  at  an  altitude 
sufficient  to  not  trouble  the  animals  while  still  making 
accurate  measurements  of  their  size  and  other  features. 
This  paper  describes  a  study  conducted  in  the  spring  of 
1996  in  the  waters  off  the  western  coast  of  the  Hawaiian 
Island  of  Maui  to  evaluate  the  utility  of  an  airborne 
Hyperspectral  Imaging  (HSI)  Sensor  to  detect  and 
recognize  whales.  This  area  serves  as  a  spring  mating 
and  calving  area  for  a  population  of  humpback  whales 
who  winter  in  Alaskan  waters.  A  Science  and 
Technology,  Inc.  (STI)  Hyperspectral  Imaging  (HSI) 
camera  was  mounted  in  a  Piper  Aztec  aircraft  moving  at 
a  ground  velocity  ranging  from  80  to  100  miles/hour.  The 
aircraft  flew  at  an  altitude  of  1000  m  above  the  whales. 
The  HSI  camera  was  operated  in  a  “pushbroom”  mode 
and  was  framed  at  a  nominal  40  frame  per  second  rate 
to  image  in  each  downtrack  frame  a  strip  on  the  sea 
surface  beneath  the  aircraft.  The  strip  was  of  width  190 
m  wide  with  aim  down  track  extent.  The  frame  rate  of 
the  camera  and  the  altitude  and  speed  of  the  aircraft 
ensured  that  successively  framed  downtrack  strips  were 
contiguous  in  space.  A  spectral  dispersion  element  in 
the  camera  divided  the  white  light  strip  image  into  72 
contiguous  spectral  bands  that  fell  on  a  Charge  Coupled 
Device  (CCD)  array  located  in  the  focal  plane  of  the 
camera.  The  system  had  sufficient  data  storage  capacity 
to  be  able  to  image  an  ocean  area  of  width  190-m  by  2 
km  in  length.  The  size  of  a  single  pixel  imaged  on  the 
ocean  surface  was  1m2.  An  internal  Global  Positioning 
System  was  used  to  reference  the  image  of  the  sea 
surface  and  the  whales  to  the  local  navigational  charts 
The  paper  gives  a  detailed  discussion  of  the  sensor, 
describes  the  spectral  image  processing  used  to  detect 
and  enhance  the  whale  images,  provides  sample 
imagery  products,  and  finally  discusses  the  potential 
utility  of  the  method  to  conduct  a  rapid  and  accurate 
whale  population  survey  at  sea. 

I.  INTRODUCTION 

In  1972  the  public  interest  in  marine  mammals  was 
expressed  by  the  Marine  Mammal  Protection  Act 
(MMPA)  passed  by  congress.  This  act  stipulates  that  the 


National  Marine  Fisheries  Service  (NMFS)  is  obligated  to 
manage  and  protect  all  marine  mammals.  In  order  to  do 
this  effectively  the  NMFS  requires  an  accurate 
population  count  of  the  different  marine  mammals.  This 
study  focuses  on  a  possible  new  counting  technique  for 
large  marine  mammals  such  as  humpback  whales. 

Besides  the  MMPA  there  are  other  reasons,  why  it  is 
important  to  know  how  many  marine  mammals  are  in 
certain  areas.  One  reason  is  that  many  experiments  are 
performed  in  the  oceans  today.  These  experiments  have 
no  intention  of  harming  animals.  A  not  so  obvious 
problem  is  to  understand  what  processes  harm  different 
animals.  In  the  case  of  marine  mammals  there  is  a  great 
uncertainty  if  noise  generated  during  an  experiment 
affects  the  animals.  To  reduce  possible  harm  it  is  better 
to  perform  all  experiments  in  areas  where  the  natural 
marine  mammal  density  is  low.  A  problem  is  to 
determine  the  location  of  these  sparsely  populated 
areas. 

Most  whale  census  methods  presently  used  are 
visually  based  counting  methods.  The  bulk  of  these 
surveys  are  conducted  from  ships  or  small  aircraft  that 
cover  the  oceans  according  to  a  statistical  line  transect 
ensuring  proper  sampling.  These  methods  are  to  a 
certain  degree  uncertain  since  they  assume  that  the 
whales  are  randomly  distributed  in  the  oceans,  which  is 
not  always  the  case.  Whales  seeking  food  are  usually 
distributed  somewhat  randomly  in  a  large  area.  This 
natural  random  distribution  is  a  close  approximation  to 
the  assumption  of  randomness  needed  for  the  statistical 
line  transect  method.  During  other  behaviors,  such  as 
migrations,  whales  congregate  in  herds  for  travel.  They 
move  large  distances  at  fairly  high  speed  making  them 
difficult  to  detect  with  the  statistical  line  transect  method. 

Another  problem  with  visually  based  methods  is  that 
the  observer  relies  on  the  animals  being  visible  from  the 
observation  platform.  This  type  of  observation  excludes 
times  when  the  animal  is  submerged  and  not  detectable 
from  the  observing  platform.  Additionally  whales 
submerge  for  a  longer  time  than  they  breathe  on  the 
surface.  The  ratio  of  surfacing  to  submersion  differs  for 
each  whale  species  and  from  animal  to  animal 
depending  on  individual  behavior.  During  food  searches 
some  species  spend  over  one  hour  submerged  with  only 
a  few  minutes  on  the  surface  between  deep  dives. 
During  migration  periods  some  whales  prefer  to  breath 
frequently,  while  others  prefer  to  travel  long  distances 
underwater.  Altogether  this  behavioral  diversity  make 
whales  difficult  animals  to  detect  with  the  presently  used 
statistical  line  transect  method. 


II.  MATERIALS  AND  METHODS 

The  data  for  this  study  was  obtained  in  February  and 
March  of  1996  in  the  ocean  basin  off  the  West  Coast. of 
Maui,  Hawaii.  This  area  is  a  winter  calving  and  breeding 
grounds'  for  Alaskan  Humpback  Whale  ensuring  that 
whales  would  be  in  the  area  of  the  experiment.  The 
sensor  was  mounted  on  a  Piper  Aztec  airplane,  which 
flew  at  speeds  of  80  to  100  miles/hour  and  an  altitude  of 
approximately  1000  m.  The  sensor  operated  in  a  push 
broom  mode  recording  72  spectral  bands 
simultaneously.  A  typical  track  segment  was  about  2  km 
long  and  190  m  wide.  The  area  resolved  on  the  sea 
surface  was  approximately  1  m2  per  individual  pixel. 

Fig.  1  shows  a  schematic  representation  of  the 
sensor  used.  The  upwelled  and  reflected  sunlight  enters 
the  sensor  through  the  entrance  slit.  The  light  is  then 
reflected  by  a  mirror  element,  and  passes  through  a 
dispersing  element.  The  dispersing  element  diffracts  the 
light  into  different  wavelengths.  These  diffracted 
wavelengths  are  then  reflected  by  a  second  mirror 
element  and  passed  onto  a  detector  array,  with  the 
spatial  axis  vertical  and  the  spectral  axis  horizontal.  The 
detector  array  is  a  Charged  Coupled  Device  (CCD) 
located  in  the  focal  plane  of  the  camera. 
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Fjg.  1:  A  schematic  representation  of  the  sensor  showing  the 
path  of  a  light  ray.  The  light  is  reflected  from  the  scan  area, 
then  passes  through  the  entrance  slit  and  is  further  reflected  by 
the  mirror  element.  The  ray  is  then  passed  through  a 
dispersing  element.  The  resulting  rays  are  reflected  again  by  a 
second  mirror  element  and  collected  onto  a  detector  array 
which  records  the  intensity  which  separates  according  to  a 
spatial  and  a  spectral  axis. 

A  geographical  positioning  system  (GPS) 
incorporated  into  the  system  obtained  images,  which 
could  be  referenced  to  a  known  latitude  and  longitude. 

III.  IMAGE  ENHANCEMENT 

Data  was  processed  on  both  PC  and  UNIX  systems 
with  the  ENVI  3.0  program.  A  processing  method  used 
to  remove  the  sea  surface  reflection  effects  from  the 


image  was  color  subtraction.  This  method  notes  that 
water  absorption  affects  blue  and  green  light  differently, 
i.e.  the  presence  of  phytoplanktonic  chlorophyll  will 
absorb  blue  light  and  reflect  green  light  while  the 
absence  of  phytoplankton  will  produce  the  opposite 
effect.  A  consequence  is  that  the  covariance  between 
blue  and  green  light  coming  from  the  sea  will  be 
dominated  by  the  Fresnel  reflection  of  the  wavy  air-sea 
interface  and  not  the  water.  This  fact  is  used  to  form  a 
composite  2-band  spectral  image  from  which  the  sea 
surface  induced  variance  is  removed.  Generally  sea 
surface  induced  variance  in  each  band  is  much  larger 
than  any  variance  induced  by  the  seawater  itself.  The 
removal  of  the  sea  surface  induced  variance  greatly 
improves  the  possibility  of  seeing  objects  in  the  water 
itself.  The  removal  operation  chooses  one  blue  and  one 
green  band,  finds  the  covariance  between  the  two 
bands,  i.e.  the  sea  surface  effect,  and  forms  a  weight  as 
the  ratio  of  the  covariance  of  the  two  bands  divided  by 
the  variance  of  the  green  band.  Subtracting  the  weighted 
green  image  from  the  blue  image  gives  a  two-band 
image  with  sea  surface  removed.  This  method  minimizes 
the  effect  of  sea  surface  and  hence  enhances 
subsurface  features  such  as  whales.  For  every  image  a 
separate  weighing  factor  was  calculated  and  used  in  the 
processing. 

Fig.  2  is  an  example  of  the  image  enhancement 
through  color  subtraction.  Fig.  2  top  represents  the 
unprocessed  image  with  the  whales  barely  visible.  Fig.  2 
bottom  is  the  processed  image  with  the  whale 
silhouettes  quite  visible.  It  can  be  seen  that  while  the 
color  subtraction  method  enhances  the  whale  silhouettes 
it  is  still  difficult  to  see  them  clearly  due  to  their  relative 
positions  to  the  sunlight.  The  whales  can  be  seen  best 
when  they  are  parallel  to  the  reflecting  light  when  their 
white  flukes  show  up  extremely  well  due  to  the  higher 
reflectivity  of  white  to  gray.  Deeper  whales  are  harder  to 
see.  Fig.  2  also  shows  the  problem  of  white  cap  clutter 
as  seen  in  both  images.  Fig.  3  top  and  bottom  also 
demonstrates  the  before  and  after  of  this  enhancement 
technique.  This  method  is  still  based  upon  visual 
inspection. 

Fig.  2:  The  top 
image  represents 
the  unprocessed 
image  where  the 
white  cap  can  be 
clearly  seen.  The 
bottom  image 
represents  the 
processed  image. 
Here  the  whales 
can  be  seen  as  well 
as  the  white  cap 


Fig.  3:  a)  the  unprocessed  image  can  be  seen.  Here  the  school 
of  whales  is  barely  visible,  b)  Shows  the  processed  image, 
where  the  whale  silhouettes  are  visible. 


IV.  A  METHOD  FOR  AUTOMATED  DETECTION  OF 
WHALES 

A  method  employing  the  dot  product  cosines  of  the 
water  and  whale  spectra  was  investigated  for  its 
potential  utility  in  the  automated  detection  of  whales. 
The  ocean  away  from  land  presents  a  much  more 
homogeneous  scene  than  does  the  typical  terrestrial 
environment.  Thus  an  image  of  the  ocean  could  be 
processed  with  a  spatial  filter  of  about  the  image  size  of 
a  typical  whale.  In  the  simplest  case  the  output  of  the 
filter  would  fall  into  two  sets:  scenes  that  looked 
spectrally  like  ocean  and  scenes  that  looked  like  whales. 
The  method  used  to  determine  this  separation  was 
developed  employing  the  dot  product  cosines  of  spectra. 
All  images,  i.e.  whale  or  water-only,  contained 
substantial  amounts  of  atmospheric  light,  sea  surface 
reflected  light,  and  light  from  the  water  itself  which 
“cluttered”  the  "whale  signal”.  Consider  the  ocean  scene 
in  Fig.  4. 


Fig.  4:  The  initial 
image  from  which  all 
samples  were 
collected.  The  true 
length  of  this  image 
is  2000m.  It  has 
been  cropped  for 
presentation 
purposes.  The  image 
width  of  190m  was 
conserved.  The  new 
length  is  400m.  A1 
through  A10  as 
labeled,  with  02  and 
05  as  the  two  water 
outliers.  wh1-wh6 
also  as  labeled  from 
top  to  bottom,  left  to 
right 

This  figure 
shows  the  areas 
where  water-only 
and  whale-only 
spectral  samples 
were  taken.  By 
inspection  of 
several  whale 
image  and  water 
image  spectra  it 


was  determined  that  spectral  differences  between  the 
two  cases  were  significant  only  for  the  spectral  region 
from  435nm  to  560nm.  Fig.  5  exhibits  the  spectra  of 


these  samples. 
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graph  shows  all  samples  with  the  average  water  spectrum 
subtracted.  The  first  ten  samples  are  the  water  area,  A1 
through  A10,  with  A2  denoted  as  02  and  A5  denoted  as  05, 
as  they  do  not  follow  the  general  trend  of  the  other  water 
samples.  The  next  spectra  are  from  the  six  whales  sample 
areas,  which  are  denoted  as  whl  through  wh6. 

Dotted  line  spectra  represent  whales  while  solid  line 
spectra  represent  water.  Note  that  whale  spectra  show 
markedly  less  blue  light  than  do  the  water-only  spectra. 
To  initiate  the  dot  product  processing  a  mean  water 
spectrum  was  formed  by  averaging  a  large  number  of 
pixels  known  to  be  free  of  whales.  This  was  done  using 


the  water-only  data  samples  from  fig.  5.  This  mean 
spectrum  contained  the  atmospheric,  sea  surface,  and 
water  light  clutter.  The  average  water  spectrum  was  then 
subtracted  from  all  other  samples  to  remove  these  mean 
attributes  from  all  spectra  both  whale  and  water-only. 
This  process  “demeaned"  the  data.  Dot  products  were 
then  formed,  by  dotting  each  whale  size  sample  whether 
water-only  or  containing  whales  against  the  average 
water  spectrum.  Dot  product  cosines  were  determined 
by  normalizing  the  dot  product  results  by  the  absolute 
scalar  magnitude  of  each  scene  and  the  mean 
background.  This  magnitude  resulted  from  the  dot 
product  of  a  spectrum  times  itself.  The  cosines  were 
transformed  into  arccosines. 

The  resulting  arccosines  were  then  considered  to 
form  a  statistical  distribution.  The  members  of  the  first 
distribution  were  the  water  only  samples.  The  members 
of  the  second  were  the  whale  samples. 

Statistical  moments,  i.e.  averages-Xave,  variances-s2 
and  standard  deviations-s,  were  calculated  for  both 
groups  and  are  shown  together  with  the  sample  size,  n, 
in  Table.  1  below. 


Tablel:  The  results  ofjhe  dot  products  for  the  al[  groups 


Water  Analysis 

Xave  = 

n  = 

sA2  = 

s  = 
DF  = 

Range  (min/max)  = 

0.025 

8 

0.00032 

0.018 

7 

0.0074 

0.043 

Whale  Analysis 

Xave  = 

0.082 

n  = 

8 

sA2  = 

0.0012 

s  = 

0.034 

DF  = 

7 

Range  (min/max)  = 

0.048 

0.12 

From  this  table  is  can  be  seen  that  the  water  and  the 
whale  groups  are  distinctly  separated  from  each  other. 
The  whale  mean  of  0.082  is  more  than  three  standard 
deviations  from  the  water  only  mean,  i.e.  0.025  +  3x.018 
=  .079  <  0.082. 

A  two-sided  Student  t-test  was  used  to  test  the 
hypothesis  that  the  two  sample  groups  were  significantly 
different  from  each  other  and  to  quantify  this  significance 
level.  The  student  t-test  was  used  because  of  the  small 
sample  size.  The  null  hypothesis  was  that  the  average 
water  dot  product  arccosine  was  equal  to  the  average 
whale  dot  product  arccosine. 

To  reject  the  hypothesis  at  the  0.005  significance 
level  (i.e.  a  1  chance  in  two  hundred  of  being  wrong)  that 
there  is  no  difference  between  the  two  populations,  the  t- 
statistic  should  fall  outside  of  the  range  of  -2.98  to  2.98. 
Since  the  observed  t  value  is  -3.88  well  outside  this 
range,  the  null  hypothesis  is  easily  rejected  and  the 


alternate  hypothesis  is  accepted  that  the  two 
distributions  are  greatly  different 

V.  CONCLUSIONS 

It  has  been  demonstrated  that  the  method  of 
hypothesis  testing  using  dot  product  arccosines  shows 
potential  for  automating  the  whale  counting  process  in 
hyperspectral  imagery  of  the  ocean.  In  a  more 
automated  method  the  mean  spectrum  of  the  water  only 
images  could  be  obtained  as  a  moving  average.  Then 
each  additional  whale  size  image  could  be  processed 
against  this  moving  average  to  form  the  arccosine  for 
hypothesis  testing.  An  advantage  is  that  the  moving 
average  will  always  be  close  to  the  environmental 
conditions,  and  no  extra  weighing  factor  need  be 
calculated  for  every  individual  image  as  required  for  the 
color  subtraction  method.  An  additional  advantage  of  this 
technique  is  that  it  does  not  rely  on  visual  inspection  for 
detection  and  hence  can  save  wear  and  tear  on  the 
census  taker. 
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